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DETAILED ACTION 



1. This Office action for US Patent Application 10/713,710 is responsive to 
communications filed 27 May 2008, in reply to the Non-Final Rejection of 28 March 
2008. Currently, claims 11, 12, 14-17, 19-24, 35, 49-53, 55-58, 60-65, and 67-70 are 
pending. Of those, claims 69 and 70 are new. 

2. In the previous Office action, the specification was objected to for an informality. 
Claim 11 was objected to for an informality. Claims 14, 15, 38, 39, 55, and 56 were 
rejected under 35 U.S.C. 112, second paragraph, for lack of antecedent basis for 
claimed subject matter. Claims 11, 12, 14, 16, 17, 20, 23, 24, 35, 36, 38, 40, 41, 44, 47- 
53, 55, 57, 68, 61 , and 64-68 were rejected under 35 U.S.C. 103(a) as obvious over US 
6, 1 1 1,604 A (Hashimoto etal.) in view of US 6, 148,031 A (Kato). Claims 13, 15, 18, 19, 
37, 39, 42, 43, 54, 56, 59, and 60 were rejected under 35 U.S.C. 103(a) as obvious over 
Hashimoto in view of Kato and ISO/IEC 11172-1 (MPEG-1 Part 1). Claims 21, 22, 45, 
46, 62, and 63 were rejected under 35 U.S.C. 103(a) as obvious over Hashimoto et al. 
in view of Kato and US 6,327,423 B1 (Ejima et al.). 



Response to Amendment 

3. Applicant's amendment to the specification is insufficient to overcome the 
objection presented in the previous Office action, but the amendment clarifies 
Applicant's original intent. It is suggested that the sentence should be amended further 
to read: "However, MPEG1 (ISO-1 172-2) standard defines that the minimum frame rate 
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is either 25 Hz for PAL or 29.97 Hz for NTSC." The current description of frame rate in 
terms of megahertz means that there are millions of frames per second. 

4. Applicant's amendment to claim 11 has been fully considered. The objection to 
claim 11 is withdrawn. 

5. Applicant's amendments to claims 11, 35, and 52 have been fully considered. 
The rejections of claims 14, 15, 55, and 56 under 35 U.S.C. 112, second paragraph are 
withdrawn. 

Response to Arguments 

6. Applicant's arguments with respect to claims 11 and 52 have been considered 
but are moot in view of the new ground(s) of rejection. Applicant states that none of the 
references previously cited disclose the claimed limitation of encoding a time period of a 
P or B picture data as the same as a time period of encoded audio data. However, it is 
respectfully submitted that it was known in the art at the time of the present invention to 
sync audio data to each frame of video data. This process is called "locked audio", and 
was known at the time of the present invention, for example, in the DV standard (IEC 
61834). 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 11, 12, 14-17, 19-24, 35, 49-53, 55-58, 60, 61, 64, 65, and 67-70 are 

rejected under 35 U.S.C. 103(a) as being unpatentable over US Patent 6,111,604 A 

(Hashimoto et al.), in view of US Patent 6,148,031 A (Kato), and in view of ISO/IEC 

11172-1 (MPEG-1 Part 1). Hashimoto et al. teaches a digital camera. Regarding 

claims 11, 35, and 52, figure 8 of Hashimoto et al. shows a block diagram of the 

camera. Image photographing section 6 comprises lens 7, lens opening 8, imaging 

element 9, and filter 10. The analog input image signal is converted in analog/digital 

converter 4 and further processed in DSP 11 (column 6: lines 40-61). This corresponds 

with the claimed "photographing means". Audio signals are input into microphone 1 and 

output through amplifier/filter 2a to analog/digital converter 4 (column 6: lines 18-26). 

This corresponds with the claimed "audio inputting means". Image data 

compression/expansion circuit 12 encodes the images from DSP 1 1 in a format such as 

JPEG or MPEG (column 6: line 62-column 7: line 2). This corresponds with the 

claimed "video encoding means" that performs the steps of "encoding the video signal" 

in the two encoding methods in claims 35 and 52. Figure 11 of Hashimoto et al. 

illustrates the process for capturing video and information. When the user presses the 

shutter button, a first picture with associated audio is captured. Image and audio files 

are stored in memory card 16, and an association file is written to link the image and 

audio files together (column 9: lines 46-54). The association file may be a container file 

for a still image, a still image with audio data, or a moving image with audio data 

(column 10: lines 1-8). However, in Hashimoto et al., no clear distinction is made 
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between various encoding modes for the recorded pictures. In the example given in 
column 9: line 55-column 10: line 4, audio may be associated with a still JPEG image, 
or a moving MJPEG image, or an MPEG image (column 6: line 65), which was known in 
the art at the time the invention was made to incorporate sound data. 

Kato teaches an image processing system in a digital camera. Regarding claims 
11 and 35, in Kato, in a continuous imaging mode, input images are initially recorded in 
real time in an intra mode as a succession of JPEG images, and stored in memory 20 
(column 3: lines 41-47). In a still image mode, the input image is recorded in memory 
20 as a single JPEG image (column 3: lines 47-53). After recording is finished, system 
control circuit 26 re-encodes the recorded series of intra images in an inter-frame 
compression mode (column 3: lines 54-63). This system control circuit corresponds 
with the claimed "controlling means" of claim 1 1 , and the selection of a still image mode 
or a motion image mode in Kato corresponds with the claimed recording mode selection 
in claims 35 and 52. 

Hashimoto et al. discloses a majority of the claimed invention except for 
encoding pictures according to two different encoding methods. Kato teaches that it 
was known to encode motion image data in a separate format than still image data. 
Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to modify the camera of Hashimoto et al. to re-encode pictures 
having a temporal aspect, such as pictures with associated sound, as inter-frame 
encoded images after encoding, as taught by Kato, since Kato states in column 2: lines 
16-34 that such a modification would enable the final recording to be achieved with 
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higher compression than with intra pictures alone, while maintaining the ability for a user 
to record a high-quality still image during the motion image recording process. 
However, while the present invention is directed to multiplexing an encoded picture 
signal and an encoded audio signal, Hashimoto et al. does not give details of its 
process of "combining" a video file and an audio file (column 1 1 : lines 34-42). 

MPEG-1 Part 1 defines the system coding layer of an MPEG-1 coded data 
stream, in which audio and video data streams are multiplexed (forward). Regarding 
claims 11, 35, and 52, Section 1-A.6.3 illustrates a sample multiplexing of a stream 
having one video and one audio stream. The stream is divided into packs, each of 
which has a header and three packets, each of 2048 bytes. First, 13 video packets are 
transmitted to ensure successful buffering. Then, an audio packet is placed for every 
6.25 video packets. Section 1-A.6.9 shows an extended sample multiplexed data 
stream. Here, a second audio packet is placed between the twentieth and twenty-first 
video packets. However, while in the shown example, one audio packet is placed for 
multiple video packets, the examiner takes Official Notice that it was known in the art for 
audio and video packets to be correlated in a 1:1 ratio, as in the "locked audio" of DV, in 
which one audio pack is present for each frame. Such a modification would ease linear 
editing to prevent mismatches between audio and video data streams at a start point or 
an end point of an edited segment. 

Hashimoto et al., in combination with Kato, discloses the claimed invention 
except for multiplexing an audio and picture signal. MPEG-1 Part 1 teaches that it was 
known to produce a multimedia datastream by multiplexing packets of audio and video 
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data. Therefore, it would have been obvious to one having ordinary skill in the art at the 
time the invention was made to implement the combination of a video and audio file in 
Hashimoto et al. as a multiplexing operation, as taught by MPEG-1 Part 1, since 
MPEG-1 Part 1 states in the Introduction that such a modification would allow for 
synchronized playback of audio and video data without having to buffer an entire sub- 
stream. 

Regarding claims 12 and 53, in Kato, "the JPEG standard is used in the still 
image compression and the MPEG standard is used in the moving image compression" 
(column 9: lines 12-14). 

Regarding claims 14 and 55, figure 12 of Hashimoto et al. shows video files and 
audio files stored in separate areas of memory card 16. 

Regarding claims 15 and 56, in Kato, as mentioned previously, video data is first 
stored in first memory 20 and then transferred to second memory 22 (column 3: lines 
54-63). This corresponds with writing multiplexed data to memory, reading the 
multiplexed data from memory, and recording multiplexed data on a recording medium. 
Additionally, in Kato, during the recording of a moving image, a still image from the 
sequence of moving images may be additionally transferred from the first memory to the 
second memory as an intra picture in an independent process of the moving picture 
recording (column 4: lines 1-10). This corresponds with encoding a video signal in the 
"first" encoding method, writing the signal to the memory, reading the signal from the 
memory, and recording the signal to the recording medium. 
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Regarding claims 16 and 57, figure 5 of Kato shows a compressed video 
encoder including DCT circuit 107, quantizing circuit 108, and variable length coding 
circuit 1 1 5 (column 5: line 60-column 6: line 24). 

Regarding claims 17 and 58, figure 12 of Hashimoto et al. illustrates audio and 
video files stored in the memory as having headers. 

Regarding claims 19 and 60, an MPEG-1 pack, containing 3 packets, is designed 
to have a pack rate of 29 Hz, or 1 frame per pack (Section 1-A.6.3). 

Regarding claims 20 and 61, Kato temporarily stores incoming data on first 
memory 20, and after re-encoding, permanently stores the data on second memory 22 
(column 4: lines 45-55, "the second memory 22 is the final storage medium"). 

Regarding claims 23 and 64, in Hashimoto et al., image data compression circuit 
12 may also perform image decoding (column 6: lines 62-66), and so corresponds with 
the claimed "video decoding means". The decoded video signals may be further 
processed in DSP 11 (column 6: lines 58-61), which outputs a video signal 26 to a 
display such as an LCD viewfinder (not shown in figure 8). This display corresponds 
with the claimed "displaying means". Digital audio signals may also be decoded in 
audio data compression/expansion circuit, transmitted to D/A converter, amplified and 
filtered in amplifier 2b, and output in output stream 26 to speaker 32 (column 5: lines 17- 
39). This corresponds with the claimed "audio outputting means". This process of 
reading data stored in memory card 16 (column 7: lines 34-50), like all other processes 
of the camera of Hashimoto et al., is controlled by CPU 23 (column 7: lines 15-16), 
which corresponds with the claimed "controlling means". 
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Regarding claims 24 and 65, the CCD in Hashimoto et al. has a resolution of 768 
x 480 pixels (column 6: line 44), and Kato inputs images at a resolution of 720 x 480 
pixels, in accordance with the NTSC standard (column 4: line 15), and produces an 
output of 320 x 240 pixels (column 4: line 24), in accordance with the CIF format. 
Although neither Hashimoto et al. nor Kato et al. record pictures at the VGA 640 x 480 
pixel standard, it would have been an obvious matter of design choice to modify the 
image sensing portion of camera of Hashimoto et al. or of Kato to produce 640 x 480 
pictures, since it has been held that a change in size of a component is generally 
recognized as being within the level of ordinary skill in the art. See In re Rose, 105 
USPQ 237 (CCPA 1955). 

Regarding claim 49, in Kato, as shown in figure 1, system control circuit 26 
controls both the image compression circuit 18, first memory 20, and second memory 
22 (column 3: lines 41-63). This system control circuit corresponds with the claimed 
"controlling means". 

Regarding claims 50 and 67, in Hashimoto et al., incoming image data from a 
camera is processed in noise reduction circuit 10 and DSP 11 (column 6: lines 40-61), 
and incoming audio data from a microphone is processed in amplifier/filter 3a (column 
6: lines 18-21). 

Regarding claims 51 and 68, in Hashimoto et al., figure 14 illustrates the 
flowchart for transmitting and receiving data from the camera to an external device 
(column 10: line 41-column 11: line 42). Data from the memory card is transferred to 
FIFO 13 (column 11: lines 25-29), and transmitted to an external device via interface 
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circuit 27 (column 7: lines 1-36). Like every other process in the camera of Hashimoto 
et al., this process is controlled by CPU 23 (column 7: lines 15-16), which corresponds 
with the claimed "controlling means". 

Regarding claims 69 and 70, in Kato, digital signal processor circuit 14 
corresponds with the claimed "first picture encoder" that performs the step of receiving a 
picture signal. In Hashimoto et al., audio data compression circuit 3 corresponds with 
the claimed "second encoder" that performs the step of receiving an audio signal. In 
Kato, image compression circuit 18 corresponds with the claimed "picture generation" 
that performs the claimed step of "generating fixed data". In Hashimoto et al., FIFO 13, 
which combines audio files and image files (column 11: lines 43-61), corresponds with 
the claimed "third encoder" that performs the claimed step of "multiplexing". 

9. Claims 21, 22, 62, and 63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hashimoto et al. in view of Kato and MPEG-1 as applied to claims 1 1 
and 52 above, and further in view of US Patent 6,327,423 B1 (Ejima et al.). Claims 21, 
22, 62, and 63 are directed to specific operations of causing a camera to perform a 
capture for a certain time. Hashimoto et al. teaches taking pictures when a shutter 
button is pressed (column 7: lines 20-24), and Kato teaches operating a keyboard to 
issue image taking commands (column 3: lines 41-53). However, the above references 
do not teach operation for a "time period" to generate data. 

Ejima et al. teaches a camcorder that records sound data. Regarding claims 21 
and 62, figure 14 is a flowchart illustrating one embodiment of the sound recording 
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control process of Ejima et al. At step S1, CPU 39 determines if a release switch 10 is 
pressed, and if it is, the image recording process begins at step S2 (column 15: line 64- 
column 16: line 3). At step S3, the sound recording process is started, and at step S4, a 
"REC" display is shown on a viewfinder to indicate that sound is being recorded (column 
16: lines 4-11). At step S5, after 10 seconds have passed, the sound recording process 
stops (column 16: lines 11-16, 34-40). However, if a sound recording switch is pressed 
within 10 seconds at step S6, sound recording continues (column 16: lines 14-23, 44- 
50). The sound recording then ends when the sound recording switch is released at 
step S20 (column 16: lines 23-50). Then, sound recording switch 12 corresponds with 
the claimed "operating means", and the time period in which the sound recording switch 
is pressed corresponds with the claimed "timing means". 

Hashimoto et al., in combination with Kato and MPEG-1, discloses the claimed 
invention except for encoding audio during the pressing of an operation means. Ejima 
et al. teaches that it was known to perform sound recording while a sound recording 
switch is pressed. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made to incorporate the sound recording switch of 
Ejima et al. into the camera of Hashimoto et al. or Kato, since Ejima et al. states in 
column 1: line 60-column 2: line 20 that such a modification would allow the timing of a 
sound recording to be independent of the timing of its associated video recording. 

Regarding claims 22 and 63, in Kato, if sound recording switch 12 is not pressed, 
then release switch 10 corresponds with the claimed "operating means", and the ten 
seconds is the "predetermined time period" in which audio is encoded. 



Application/Control Number: 10/713,130 



Art Unit: 2621 



Page 12 



Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US Patent 5,572,333 A (Moriyama et al.) teaches a video 
recording method that places video frames into packets of a fixed number of frames. 
US Patent 5,715,356 A (Hirayama et al.) teaches a system for generating a compressed 
multimedia bitstream in which video data and audio data is placed within "groups" of a 
fixed number of image frames. US Patent 5,758,180 A (Duffy et al.) teaches a video 
editing system that adjusts the synchronization between audio and video data. US 
Patent 6,108,486 A (Sawabe et al.) teaches an audio recording system that divides an 
audio track into "frame" units inside audio packs. "DV Data in the AVI File Format 
Specification" shows that the DV format was known in the art at the time of the present 
invention. 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 
1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner, whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessfully, the 
examiner's supervisor, Mehrdad Dastouri, can be reached on 571-272-7418. The fax 
phone number for the organization where this application or proceeding is assigned is 
571-273-8000. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/D.N.WV 

Examiner, Art Unit 2621 
/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 
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